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Method, System and User Terminal for Collecting Information 
on Audience of Broadcast Media Stream 

Field 

The invention relates to a method, a system and a user terminal for 
5 collecting information on the audience of a broadcast media stream. 

Background 

Many broadcasters are interested in monitoring their audience for 
several reasons. Especially commercial radio or television broadcasters need 
customer information for improving their business; the more they have listeners 
the more tempting they are from an advertiser's point of view. Advertisers also 
need information on their campaigns: how many customers listened to an ad* 
vertisement, whether they liked it or not, how well the product sold, etc. 

In the prior art there are several research methods for collecting in- 
formation on audience. Some widely used methods are questionnaires or fo- 
cus group interviews. These methods are quite slow and they cannot be used 
in online adaptation of the broadcast. Additionally, questionnaires or group in- 
terviews are directed to a selected audience group, the size of which is limited. 
Therefore the reliability of the results depends strongly on advance planning 
and the size of the group, since the size of the group has to be quite significant 
in order that the result is reliable, the expenses rise correspondingly. 

Especially, the prior art methods are not rapid enough in monitoring 
the audience of interactive programs which nowadays are becoming more and 
more popular. 

Brief description of the invention 

25 An object of the invention is to provide an improved method for col- 

lecting information on an audience of a broadcast media stream. The method 
comprising: connecting a radio telecommunication system and a broadcast 
system to perform a media system, broadcasting a media stream on at least 
one broadcast channel of the broadcast system, transmitting parallel informa- 

30 tion on at least one parallel channel which parallel information Is associated 
and synchronized with the media stream of at least one broadcast channel, 
connecting the audience to the media system as passive users or active users, 
the active users receiving the parallel information and the passive users not 
receiving the parallel information, collecting information on the audience con* 
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nected to the media system, the infomiation being on active users and/or pas- 
sive users, processing the collected information in a predetermined way. 

According to an aspect of the invention, there is provided a media 
system for collecting information on the audience of a broadcast media stream, 

5 comprising: means for broadcasting a media stream on at least one broadcast 
channel of the broadcast system, the broadcast system being part of the media 
system, transmitting parallel information on at least one parallel channel which 
parallel information is associated and synchronized with the media stream of at 
least one broadcasting channel using a radio telecommunication system of the 

10 media system, means for connecting the audience to the media system as 
passive users or active users, the active users receiving the parallel informa- 
tion and the passive users not receiving the parallel information, means for 
collecting information on the audience connected to the media system, the in- 
formation relating to active users and/or passive users, means for processing 

15 the collected information in a predetermined way. 

According to another aspect of the invention, there is provided a 
user terminal for collecting information on an audience of a media stream 
broadcast by a media system comprising: means for receiving parallel informa- 
tion on at least one parallel channel which parallel information is associated 

20 and synchronized with the media stream of at least one broadcast channel of 
the broadcast system, the broadcast system being a part of the media system, 
means for connecting the audience to the media system as passive users or 
active users, the active users receiving the parallel information and the passive 
users not receiving the parallel information, means for changing a parallel in- 

25 formation reception state according to the user's selections, means for trans- 
mitting selections made by active users for collecting information on the audi- 
ence connected to the media system, means for showing the collected infor- 
mation. 

Further embodiments of the invention are described in the depend- 
so ent claims. 

The method and system of the invention provide several advan- 
tages. According to an embodiment of the invention, the radio station is able to 
observe the development of an on-line audience almost real time, obtain view- 
able statistical infonmation on the audience and arrange interactive polls, com- 
35 petitions, etc. 
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List off drawings 

In the following, the invention will be described in greater detail with 
reference to the preferred embodiments and the accompanying drawings, in 
which 

5 Figure 1 shows an example of a wireless telecommunication sys- 

tem, 

Figure 2 is an example of a media system, 
Figure 3 is a flow chart, 

Figures 4A-C show an example of a broadcast system, a server and 
10 a content creation tool, 

Figure 5 shows an example of a user terminal. 

Description of embodiments 

Referring to Figure 1, a blocl< diagram illustrates an example of a 
wireless telecommunication system. The wireless telecommunication system 
15 may be based on, for example, UMTS (Universal Mobile Telephone System) or 
WCDMA (Wideband Code Division Multiple Access). 

The core network may correspond to a combined structure of the 
GSM (Global System for Mobile Communications) and GPRS systems (Gen- 
eral Packet Radio Service), for example. The GSM network elements are re- 
20 sponsible for the implementation of circuit-switched connections, and the 
GPRS network elements for the implementation of packet-switched connec- 
tions, some of the network elements, however, being shared by both systems. 

A centre 100 represents a mobile services switching centre (MSC) 
and a serving GPRS support node (SGSN) enabling circuit-switched and 
25 packet switched signalling, respectively, in the radio system. 

The core network may have a gateway unit 102, which represents a 
gateway mobile service switching centre (GMSC) and a gateway GPRS sup- 
port node. The GMSC attends to the circuit-switched connections between the 
core network and external networks, such as a public land mobile network 
30 (PLMN) or a public switched telephone network (PSTN), and the GGSN at- 
tends to the packet-switched connections between the core network and ex- 
ternal networks such as the Internet. 

The centre 100 controls a radio access network (RAN) 104, which 
may comprise at least one base station controller 106 controlling at least one 
35 base station 108. The base station controller 106 may also be called a radio 
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network controller and the base station may be called a node B. A subscriber 
terminal 110 communicates with at least one base station 108 over a radio in- 
terface. 

The radio server 112, which is an important part of the present solu- 

5 tion, may be connected to the centre 100, but it may also be connected to the 
gateway 102 or to some part of the RAN 104. In some applications, the radio 
server 112 may communicate with the radio system over a radio interface in 
the same manner as the subscriber terminal 110. 

Telecommunication systems are explained in further detail in the 

10 standards and literature of the field. 

Figure 2 shows an example of an embodiment of how a wireless 
telecommunication system and a broadcasting system are connected together 
for performing a media system for transmitting parallel information related to a 
broadcast media stream, such as a radio program. Regarding the parallel 

15 channel, the media system may comprise the following elements: a broadcast 
system 200, a content creation tool 202, a server 112, the network 104 of a 
wireless telecommunication system and the subscriber terminal 110. The 
broadcast system 200 of a broadcast station uses a digital content manage- 
ment system to run a broadcast, such as an FM transmission, an AM trans- 

20 mission or a digital radio or television transmission. The broadcast system 200 
broadcasts TV or radio programs. 

The content creation tool 202, which may also be called a visual ra- 
dio tool, is used to create parallel channel content presentation to be displayed 
on subscriber terminals. The content creation tool 202 typically locates in the 

25 radio or television station and it may be integrated into the broadcast system 
200. 

The server 112 provides the users with the content related to the 
broadcast based on their current parallel channel selection. The server 112 
controls the content flow to and from the subscriber terminals. It facilitates a 

30 timed delivery of the content to the subscriber terminal, as well as the collect- 
ing and forwarding of interaction results to the radio station. The server con- 
trols the number of users. If necessary, the server limits the number of users 
using the parallel channel at the same time. The server also takes care of con- 
tent adaptation for different application platforms in various subscriber termi- 

35 nals. 
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A radio access network (RAN) 104 provides the connection to the 
subscriber terminals via a wireless telecommunication network. The sub- 
scriber terminal may be a mobile phone, a palm computer or a laptop, for in- 
stance. The details of the radio access network and subscriber terminals may 
5 vary according to the manufacturer and the used wireless telecommunication 
standard applied. 

Figure 3 is a flow chart depicting a preferred embodiment of a 
method for collecting information on the audience of a broadcast media 
stream. An embodiment of the method uses a visual radio application in the 

10 subscriber terminal, a visual radio server and a content creation tool, which will 
be discussed later in this application. The method is suitable for obtaining in- 
formation on the audience of a radio or TV program for example to poll audi- 
ence opinions in relation to a broadcast (programs or advertisements), to cre- 
ate interactive advertising and other interactive events, such as voting and 

15 quizzes, or to keep record of the number of the audience in general. The radio 
station is able to observe the development of an on-line audience almost real 
time, obtain statistical information on the audience and arrange interactive 
polls, competitions, etc. 

The method begins in block 300. In block 302 a telecommunication 

20 system and a broadcasting system are connected together for performing a 
media system for transmitting parallel information related to a broadcast media 
stream. The media system is depicted in more detail in Figures 2 and 4A-C. 
The parallel information (also called the parallel channel content) typically com- 
prises one or more parallel information objects (text strings, graphic file ob- 

25 jects, animations, video clips, etc) and instmctions for displaying the objects 
(positions of objects, timing, etc.). 

In block 304 a media stream is broadcast on at least one broadcast 
channel of the broadcast system. Nowadays, the broadcast system of a broad- 
cast station typically uses a digital content management system to run a broad- 

30 cast, such as an FM transmission, an AM transmission, or a digital radio or 
television transmission. Programs including advertisements and breaks for ad 
hoc presentations are stored beforehand and the digital content management 
system runs the programs under the control of an editor. 

In block 306 parallel information which is associated and synchro- 

35 nized with the media stream of at least one broadcast channel is transmitted 
using the media system. The parallel information is transmitted on a parallel 
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channel. There may also be several parallel channels. The process regarding 
the creating and transmitting of the parallel information will be discussed in 
further detail later in this application. Briefly, the parallel information is created 
using the broadcast system, the content creation tool and the visual radio 

5 server and then sent to the user via a radio telecommunication network. The 
parallel information is usually transmitted as a packet transmission. 

In block 308 the audience is connected to the media system either 
as passive users or active users, the active users receiving the parallel infor- 
mation and the passive users not receiving the parallel information. In other 

10 words, the term active refers to users who are actively receiving the content 
updates delivered by the radio server, i.e. these users are in a flow mode. Pas- 
sive users are users who are not receiving the content updates, and may or 
may not be using the service at a particular moment. The audience has pref- 
erably an interactive connection to the media stream system via an interactive 

15 menu shown on the screen of their user terminals. The users who have the 
terminal application running, and are viewing the (delivered parallel channel) 
content on the screen are active users. The users who have the application 
running but do not receive the content are passive. A user might not receive 
/see the content due to multiple reasons: the application is minimized or run in 

20 a background, another application is running on the screen foreground and 
application has no display screen, screen saver is on, the application is run- 
ning but a user is doing something else which prevents him seeing the parallel 
channel (i.e. browsing some other content). The application should report 
changes in the state to the server. 

25 In block 310 information on the audience connected to the media 

system is collected. This is done by using the parallel channel as a feedback 
channel. If the system comprises several parallel channels, it is possible to use 
one channel as a fon^/ard channel and another as a feedback channel, for in- 
stance. The information may relate to active users and/or passive users. The 

30 state of parallel information reception (active or passive) of a user terminal is 
reported to the server to provide on-line information on the number of active 
and/or passive users. The server collects the information and maintains a da- 
tabase of the statistics. In addition, the information may relate to polling audi- 
ence opinions in relation to broadcasts (programs or advertisements). Informa- 

35 tion may be collected from advertising, votes and quizzes, etc. For interactive 
advertising, the number of responses received from the audience is collected. 
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The templates for the screen menu of a subscriber terminal may be 
ready-made, in which case they are stored in a user temilnal and modified ac- 
cording to the parallel information. The templates may also be transmitted as 
parallel infomiation. 

5 In an embodiment of the method, the subscriber terminal reports to 

the radio sen/er when the application mode, that is the parallel information re- 
ception state, changes. Only users in the flow mode (active users) are as- 
sumed to be able to view the current content, whereas other (passive users) 
may be listening to the broadcast but are not actively following the information 

10 on the parallel channel. When a user is in the flow mode, content delivered 
from the server is updated automatically on the terminal screen according to 
the timing definition. The application may be switched to the background or to 
the initiating communication off-mode or a screen saver may be turned on, pre- 
venting the user from viewing the current content updates. These users are 

15 called passive users. 

The events to be reported to the server are, for instance: an applica- 
tion start, i.e. a currently selected radio station is reported to the server, an ap- 
plication end, i.e. a pause in the reception of a parallel channel and or a 
change of a mode of the application is reported (for instance from a pause 

20 state to a resume state), and an application poll, i.e. a response to a query 
sent by the server is reported. The server may also update its information peri- 
odically. 

The visual radio application may be instructed to send a response to 
the server when the user starts an interactive connection by a pressing a vir- 
25 tual button on the screen. The response to be sent is defined separately for 
each button on the screen by the content creation tool. The timing of the 
chances in the general appearance of the screen may be defined to match a 
particular event in the broadcast, such as a song, an advertisement or a com- 
petition. 

30 The subscriber temninal may have an identifier that is sent to the 

sen/er with an event report. This dentifier may be a terminal-specific identifier 
or derived from one, e.g. IMSI or MSISDN, or created by the sender as a re- 
sponse to a terminal request and stored by the terminal between sessions 
(similar to the cookie mechanism in the Internet). This makes it possible to ob- 

35 tain information on a particular subscriber terminal. An interactive event, such 
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as a quiz, may also have an identifier that enables connecting answers to the 
corresponding interactive event and performing statistics. 

In block 312 the collected information is processed in a predeter- 
mined way, for example statistically, The information may also be shown on 

5 the screen of a user terminal or a broadcaster's terminal. Templates for show- 
ing the collected information may be made and stored in a terminal in advance. 
The ready-made templates can be modified according to the parallel informa- 
tion. If the templates do not require too much transmission capacity, they can 
also be transmitted entirely as a parallel information. 

10 In an embodiment, the server maintains a database for subscriber 

terminal event reports and uses the database to create graphics of statistical 
information, which are shown in a subscriber terminal using a content creation 
tool. The graphics may be automatically updated when new information be- 
comes available or created as a response to a query. Many kinds of informa- 

15 tion views may be shown on the basis of events reported from subscriber ter- 
minals: the number of current users, current active users and current passive 
users. The number of current users is increased when an application start- 
message is reported and decreased when an application end -message is re- 
ported. The number of current active users is increased when an application 

20 start or an application resume -message is reported and decreased when an 
application pause or an application end -message is reported. The number of 
current passive users is increased when an application pause is reported and 
decreased when an application resume is reported. It should be noticed that 
application polling and periodic reporting can be used to refresh the numbers 

25 at any time. 

When it comes to the interactive advertising, the number of re- 
sponses received from the audience is collected. When an interactive screen 
containing different options is delivered to subscriber terminals, graphics 
and/or other corresponding information (options, images, etc.) are initialised, 

30 Each time an interactive response is reported from a subscriber terminal as a 
selection preformed by the user, the number of the option is increased in the 
database of the server. The share of the audience participating in the action in 
question can be given as a percentage of selected options compared with the 
number of active users, for instance. The information in the database of the 

35 server can be saved for later use to enable long-term analysis of the data. 
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A couple of examples of utilising the information gatiiered fonn the 
audience are: the collected information may be used for searching for good 
hours, geographical areas and/or modes for advertising; the best time to ad- 
vertise is, of course, the time when the amount of the audience is biggest. The 
5 type of the audience is also important and this is why the user profiles are im- 
portant. The geographical areas are interesting, when the advertiser wants to 
aim his advertising to a particular area, its neighbourhood, for instance. It is 
also possible to improve the mode of advertising by using polls. 

Because each event reported from a subscriber terminal contains 
10 an identifier, it is possible to use the database for a statistical analysis. One 
example is the forming of listening profiles, for example the number of times a 
user returns to the station during a predetermined period of time. 

If the user identifier is compared to demographic statistics or other 
comparable information on the user, the database can also be used for more 
15 specific analysis of user groups and segments in real time. 

The method ends in block 314. An arrow 316 depicts one possibility 
for repeating the method. 

It should be noticed that it is possible to collect location information 
on the audience. For this purpose, a user temiinal may send information on its 
20 location or there may be a separate database comprising location infomiation. 
By using location information it is possible to obsen^e the amount of audience 
in a particular location. 

Next, an example of an embodiment of the method for collecting in- 
formation on the audience of a broadcast media stream is discussed. Although 
25 this embodiment is described with reference to a radio broadcast systems. The 
invention can be applied to other broadcast systems as well. 

The transmitting of parallel information associated and synchronized 
with the media stream of at least one broadcasting channel comprises two 
main phases: creating parallel information and transmitting it. 
30 The parallel information is created as follows: a radio station creates 

a parallel information definition message by using the content creation tool 
202. The creating of the parallel information definition message requires infor- 
mation such as user options, a displayed prompt text, a screen layout, a paral- 
lel information identifier and timing information. The content creation tool 202 
35 sends the parallel information definition message to the server 112. The server 
112 searches for a corresponding parallel information object in the database. 
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The server 112 transmits the parallel information object usually as a packet 
transmission to the connected temninals 110. The parallel infomnation object is 
saved in the memory of a subscriber terminal 1 10 for later use, or it is shown 
immediately. The server 112 may modify the parallel information object by de- 
5 fining terminal-specific parameters, such as virtual button sizes and positioning 
on the screen of the subscriber terminal and/or graphics according to the re- 
quirement of the subscriber terminal in question. The parallel Information is 
shown on the screen of the subscriber terminal in synchronization with the 
broadcast. 

10 Information on active users is collected as follows: a user answers 

to a poll, quiz or the like, by selecting an option shown on the screen by press- 
ing a virtual button. The answer, and together with it, the user identification 
and/or the parallel information identifier are communicated to the server 1 12. 

The information can be processed in several ways. For example, 

15 the server 112 aggregates the number of the answers corresponding to the 
answer in question. 

The content creation tool periodically inquiries the server to get up- 
dated statistics. The updated statistics are read from the database of the 
server 112 and sent to the content creation tool 202 which updates its display 

20 in such a way that the current situation can be seen. The content creation tool 
202 is in this example the terminal of the broadcaster. 

Figures 4A-B depict more details of the broadcast system, the 
server and the content creation tool. The shown functional structures are ex- 
amples and It is obvious for a person skilled in the art that the broadcast sys- 

25 tem, the server and the content creation tool can be implemented In several 
ways. 

The broadcast system is shown in Figure 4A. The broadcast system 
200 comprises a timing information module 404, a dynamic content delivery 
module 402 and a user interface 406. Timing information is used to synchro- 

30 nize transmissions of the broadcast channel and the parallel channel. It is 
communicated to the server 112 by the timing information module 404. The 
timing information module 404 provides information on the starting time and 
ending time of a particular program, as well as the timing of advertising breaks 
etc. The length of broadcast advertisements and similar breaks can be de- 

35 duced, for example, from the show's run time at the starting time of the break. 
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The dynamic content delivery module 402 can feed additional con- 
tent infomriation to a content structure tool 416 and a content packaging mod- 
ule 414 included in the content creation tool 202 and, finally, to a content de- 
livery engine 410 in the server 112. The additional content information from the 

5 dynamic content delivery module 402 can be, for example, results of events 
arranged in the broadcast, such as the name of a winner in a quiz show. When 
the additional content information is determined during the program, for exam- 
ple, during live events, a sports coverage or in a broadcast radio, the additional 
content information is communicated to the content packaging module 414 in 

10 the content creation tool to dynamically create additional content items of the 
parallel information. 

In addition, the dynamic content delivery engine module 402 may 
receive interaction results from an interaction engine 412 located in the server 
112 through a feedback module 418 in the content creation tool 202 and to be 

15 used as a part of the broadcast, for example to display the results of a vote on 
the TV as a video overlay. 

A user interface 406 controls and adjusts by the broadcasting per- 
sonnel the parallel channel timing with respect to the broadcast media stream. 
For example, signalling in the parallel channel may be paused and resumed to 

20 stop and resume the playing of the parallel channel. These control events are 
communicated through a synchronization engine 408 located in the server 112 
to the subscriber terminal 110, which then adjusts parallel channel timing ac- 
cordingly. 

The broadcast content delivery module 400 supplies the broadcast 
25 to a broadcast receiver, such as an FM radio receiver, a TV set, or the like. 
The receiver may be situated in the subscriber terminal. 

The feedback module 418 of the content creation tool 202, the in- 
teraction engine 412 and the memory 420 of the server 112 process the inter- 
action results and create a suitable presentation to be shown to the broadcast- 
30 ing personnel or to the receivers of the broadcast. 

The server is shown in Figure 4B. The server 112 provides the us- 
ers with content related to the broadcast based on their current parallel chan- 
nel selection. The server controls the flow of the parallel information content to 
and from the subscriber terminal. It facilitates the synchronized delivery of con- 
35 tent to the subscriber terminal as well as the collecting and forwarding of inter- 
action results to the radio station. The server controls the number of users. If 
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necessary, the server limits the number of users using the parallel channel at 
the same time. The server also takes care of content adaptation for different 
application platforms in various subscriber terminals. 

To enable statistics on both currently active and passive users to be 

5 complied, the subscriber terminal 110 reports to the server 112 when the ap- 
plication mode, that is the parallel infonmation reception state, changes. Only 
users in a flow mode (active users) are assumed to be able to view the current 
content, whereas other (passive users) may be listening to the broadcast but 
are not actively observing the parallel channel. When a user is in the flow 

10 mode, content delivered from the server is updated automatically on the termi- 
nal screen according to the timing definition. The application may be switched 
to the background, to the initiating communication off-mode or a screen saver 
may be activated, preventing the user from viewing the current content up- 
dates of the parallel information. 

15 The synchronization engine 408 receives the starting time and the 

advertising break information from the timing information module 404 of the 
broadcast system 200. In addition, the synchronization engine 408 provides 
the means for subscriber terminals to synchronize their clocks with broadcast 
system time references by running a synchronization algorithm with the server. 

20 The content delivery engine 410 delivers a signal with the content 

package created by the content packaging module 414 of the content creation 
tool to the content delivery engine 410 in the server. The signal is fed from the 
content delivery engine 410 to the interaction engine 412 in the server which 
sends the signal through the network 104 to the subscriber terminal 110 as 

25 parallel information. A subscriber-terminal-specific variant of the content can 
be delivered, which contains graphics objects optimised to the capabilities and 
the screen size of each subscriber terminal. A content package, including ob- 
ject identifications or other content items, may correspond to an entire program 
and be delivered before the program starts. The content can alternatively be 

30 delivered in a single package that contains both the subscriber terminal soft- 
ware module in a suitable format, such as a Java Ml Diet and the content pack- 
age for a particular show. This is suitable for recorded shows and programs, 
where the content and content timeline are known beforehand. In that case, 
dynamic content can be delivered in addition to the content package to take 

35 care of variation in the content during the broadcast. 
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Alternatively, the content can be delivered dynamically in content 
blocks, one block corresponding to a segment of a program. This approach is 
suitable for FM radio for example, where several songs to be played and dif* 
ferent kinds of breaks such as advertising breaks are usually decided in ad- 
5 vance. 

If a user answers to a poll, for instance by selecting an option shown 
on the screen by pressing the virtual button in question, the user identification 
and/or the parallel information identifier are communicated together with the 
answer to the interaction engine 412 of the server. The server aggregates 

10 number of the answers relating to the poll. Therefore, the server also com- 
prises a memory 420, where identifiers and answers are stored. A memory 
record may be composed of a parallel information identifier, typically a number, 
for associating answers with the corresponding poll, advertisement etc, and the 
number of selections of each option, for example, and/or the memory record 

15 may be composed of a user identifier for forming a user profile. 

The content creation tool 202 may periodically request the server 
112 to give updated statistics. The updated statistics are read from the data- 
base and sent to the content creation tool 202, which updates its display in 
such a way that the current situation can be seen by a broadcaster. 

20 If the responses are in text format, the radio server saves the re- 

sponses In text format. The content creation tool then receives the responses 
in the text format and they are shown on the display as such. 

The server 112 maintains a database in a memory 420 for sub- 
scriber terminal event reports and uses the database to create graphics of sta- 

25 tistical information, which are shown on a subscriber terminal (or on a broad- 
caster's terminal using a content creation tool 202). The graphics can be up- 
dated tained automatically when new information becomes available or created 
in response to a query. 

Because each event reported from a subscriber terminal contains 

30 an identifier, it is possible to use the database for analysing of user profiles, for 
example the number of times the user returns to the station during a predeter- 
mined period of time. 

if the user identifier is compared to demographic statistics or other 
comparable information on the user, the database can also be used for a more 

35 specific analysis of user groups and segments in real time. 
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The content creation tool is shown in Figure 4C. The content crea- 
tion tool 202, which may also be called a visual radio tool in many applications, 
is used to create the parallel channel content (or a parallel information object) 
presentation to be displayed on the subscriber terminal. The content creation 

5 tool can be used to poll audience opinions relating to broadcasts (programs or 
advertisements), to create interactive advertising and other interactions, such 
as votes and quizzes. The content creation tool is also used to monitor the 
compiled statistics on the server and to create or modify the outer appearance 
of the display of a subscriber terminal. It is also used to create the content of 

10 the parallel channel for collecting user responses. The parallel channel acts as 
a feedback channel for user responses sent back from subscriber terminals. 

The content creation tool is typically located in the radio or television 
station and the content creation tool may be integrated into the broadcast sys- 
tem 200. A content structure tool 416 creates timing for displaying parallel 

15 channel content associated with a broadcast media stream in time (for exam- 
ple, to show this parallel channel object on the screen of the subscriber termi- 
nal at 14:43:02 after the beginning of the program). Responses that are sent to 
the server 112 in response to the user interaction can be fed to the feedback 
module 418 located in the content creation tool. 

20 The content stmcture tool 416 defines the layouts for displaying the 

parallel information objects, such as the sizes and positions of the objects on 
the screen of the subscriber terminal. 

The content packaging module 414 is used to create a content 
package I.e. the fonmat in which the parallel information is delivered to the sub- 

25 scriber terminal 110, including instructions for displaying the information (posi- 
tions of objects, timing, etc.), as well as the content to be displayed, which 
usually consist of one or more parallel information objects (text strings, graphic 
file objects, animations, video clips, etc). 

Generally, the content creation tool allows the radio station to create 

30 a visual presentation and manage the content flow shown on the screen of the 
subscriber terminal in synchronization with the broadcast. In addition, the con- 
tent creation tool allows the station to manage interactive elements, such as 
delivery and purchase of objects, votings and quizzes. 

Additionally, it is possible to utilise location information on the audi- 

35 ence. This is typically done by compiling location information in the server 
which then communicates the information to the content creation tool. The con- 
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tent creation tool visualises by using a map the amount of the audience in a 
particular area, for instance. The Infomriation can be used for profiling the audi- 
ence, designing advertising, for instance. The user terminal may send informa- 
tion on Its location or there may be a separate database comprising location 

5 information. Typically, it is not required to follow a particular audience member 
but to see the amount of audience in a determined area. 

Figure 5 shows a simplified example of a subscriber terminal 
whereto the embodiment of the invention can be applied. The terminal may be 
a mobile telephone or a microcomputer, for example, without being restricted 

10 thereto. 

In the subscriber terminal, the media system provides an integrated 
user experience, allowing the user to start both listening to a broadcast and 
receiving the content relating to the broadcast on the parallel channel. The op- 
eration of the media system is based on two channels, the broadcast channel 

15 and the parallel channel. The media system, on the other hand, provides a vis- 
ual and interactive parallel channel functionality, such as receiving and display- 
ing content objects and reporting user interactions to a server. 

The terminal comprises an antenna 500 with which signals are both 
transmitted and received via a duplex filter. A terminal may also comprise a 

20 broadcast receiver 518 such as a TV or a radio tuner, a video streaming en- 
gine, etc. for receiving for example radio or TV programs, with which case 
there is a need for a broadcast receiver antenna. These antennas may be in- 
tegrated or separate antennas. 

The terminal further comprises a transmitter for a wireless telecom- 

25 munication system 502. to amplify and transmit a modulated signal to the an- 
tenna, a modulator 504 modulating the carrier wave by a data signal compris- 
ing the desired information in accordance with a selected modulation method, 
a receiver 506 which amplifies the signal supplied from the antenna and down- 
converts the signal to a selected intermediate frequency or directly to base- 

30 band, and a demodulator 508 demodulating the received signal to enable a 
data signal to be separated from the carrier wave. 

The subscriber terminal also comprises a controller block 516 com- 
prising, for example, control and calculation means for controlling the operation 
of the different parts of the terminal, means for processing the speech of the 

35 user or the data generated by the user, such as a digital signal processing 
(DSP) processor comprising, for example, channel conrection functions com- 
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pensating for interference in the signal caused by the radio channel by utilising 
information on the channel obtained from a known training sequence, A/D con- 
verters converting an analogue signal into a digital one by sampling and quan- 
tizing the baseband signal, D/A converters converting a digital signal to an 

5 analogue one by a reverse method, filters at the receiver which filter frequen- 
cies outside a desired frequency band or, which in band-restricted systems 
restrict the band width of the output at the transmitter, and coding and decod- 
ing means for both channel and speech coding. 

Furthermore, in spread-spectrum systems, such as WCDMA, the 

10 spectrum of the signal is spread at the transmitter by means of a pseudo- 
random spreading code over a wide band and despread at the receiver, in an 
attempt to increase channel capacity. The control block also comprises means 
for arranging the signal to be transmitted and the signalling information to con- 
form to the air interface standard of the cellular radio system used. 

15 The user interface of the terminal comprises a loudspeaker or an 

earpiece 510, a microphone 512, a display 520 and possibly a keypad and/or a 
joystick or a similar device. The user interface devices communicate with the 
control block. 

The temninal also comprises several different memory elements that 

20 are shown as one functional block 514. The parallel information delivered to 
the subscriber temninal from the server interaction engine 412 may be stored in 
the local memory 514. 

The content structures (the layouts and other display instructions for 
displaying the parallel information objects typically performed on a slide basis, 

25 where the slide means one screen of information in the subscriber terminal) 
are separated from the parallel information and saved to the content structure 
memory 834. The content structure also comprises possible templates. The 
content objects (for example, texts and figures to be shown on the screen), in 
other words the parallel information objects are also separated and saved to 

30 the content object memory 532. The separation and saving can be performed 
as a background process, whereby the required information is always available 
to a content processor 522 and a rendering engine 524 when needed. The 
rendering engine 524 forms visual and acoustic effects. Default information 
can also be stored in the local memory to be shown in case dynamic content 

35 cannot be delivered to the subscriber terminal in time. 
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The initiation of the parallel channel can be made in several ways. 
The user can select an operation which causes the subscriber tenminal to re- 
ceive and display the infomnation from the parallel channel. If the subscriber 
terminal includes a receiver for the broadcast channel, the user can select a 
5 suitable broadcast channel and the subscriber terminal may automatically initi- 
ate the reception and display of the parallel channel, instead of manual initia- 
tion. 

When a parallel channel software operation is initiated in the sub- 
scriber terminal, he subscriber terminal may transmit information about itself to 

10 the server 112 and the server informs the subscriber terminal of the media 
stream. With the information on the subscriber terminal, a specific variant of 
the content can be delivered, which contains graphics objects optimised to the 
capabilities and the screen size of each subscriber terminal. 

A timer controller module 528 establishes a connection to the 

15 server. The timer controller module 528 runs a synchronization algorithm to 
synchronize an intemal clock of the subscriber terminal with the time in the 
server. A simple synchronization algorithm can be used, based on calculating 
round trip delays of requests sent to the server from the subscriber terminal 
and calculating the difference between the subscriber tenminal clock and the 

20 server clock. Once the subscriber terminal has performed the synchronization 
and the starting time of a program is known, media stream timeline references 
can be translated to corresponding of references in the intemal clock of the 
subscriber terminal. The parallel information can be shown to the user in syn- 
chronization with the broadcast. 

25 The timer controller 528 determines whether the program has al- 

ready started and what is the current timeline position. If the program is run- 
ning, the timer controller 528 can automatically find the correct content item in 
the parallel channel to be displayed in the display of the subscriber terminal. 

Once the software of the subscriber terminal is activated and the 

30 subscriber terminal has received parallel information to be shown, the content 
processor 522 starts executing. The processor reads the content structure in 
the content structure memory 534 for determining the current content to be 
displayed and communicates the content structure to the rendering engine 
524. Based on the content structure, the content processor 522 informs the 

35 timer controller 528 to trigger proceeding in the parallel channel according to 
the content structure. Then the content processor 522 is initiated again to dis- 
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play the next parallel information object (content item). The processor 522 
reads another content structure in the content structure memory 532, deter* 
mines the next parallel information object and possible layouts and sends them 
to the display 520. 

5 The subscriber terminal sends an action report message to the 

server, when a change in the parallel information channel state is triggered or 
a user answers to a poll, etc. The events to be reported to the radio server are, 
for instance: an application start (active state), i.e. a currently selected radio 
station is reported to the server, an application end, i.e. an application pause is 

10 reported when the mode of application changes (active/passive), an applica- 
tion poll, i.e. a response to a query sent by the server is reported and a peri- 
odic reporting. 

A signal having information on the action is communicated to the 
content processor 522. The action may be a report on a change in the parallel 

15 information reception state or on a user activity such as an answer to a quiz, 
for example. The content processor then analyses the action and triggers a 
transaction as defined by the content structure of the action. The user activity 
automatically triggers communication from the subscriber terminal 110 to the 
server 112 and thus the user does not need to know a long list of service num- 

20 bers, service codes or identification strings. The signal of the user action is 
communicated from the content processor to the interaction engine 526 of the 
subscriber terminal, which transmits the signal through the network to the in- 
teraction engine 412 of the server using the parallel channel as a feedback 
channel. 

25 A subscriber terminal may have an identifier that is sent to the 

server with an event report. This identifier can be a temninal-specific identifier 
or derived from one, e.g. IMSI or MSISDN, or created by the server in re- 
sponse to a terminal request and stored by the terminal between sessions 
(similar to the cookie mechanism in the Internet). This makes it possible to ob- 

30 tain information on a particular subscriber terminal. An interactive event, such 
as a quiz, may also have an identifier that enables connecting answers to the 
corresponding interactive event and compiling statistics. 

Blocks 522, 524, 526 and 528 required for the parallel channel com- 
munication may be situated in a specific software module. This module can be 

35 implemented using an operating system, such as Symbian, or a programming 
environment, such as Java MIDP. 
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After the parallel channel software module in the terminal is initiated, 
the module is informed of the broadcast and/or the program the user is inter- 
ested to follow. The user can also make the selection by using another termi- 
nal software module, such as a program guide delivered to the terminal over a 
5 browser interface. 

Even though the invention is described above with reference to an 
example according to the accompanying drawings, it is clear that the invention 
is not restricted thereto but it can be modified in several ways within the scope 
of the appended claims. 



10 
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Claims 

1 . A method for collecting information on an audience of a broadcast 
media stream, characterized by 

(302) connecting a radio telecommunication system and a broad- 
5 cast system to perform a media system , 

(304) broadcasting a media stream on at least one broadcast chan- 
nel of the broadcast system, 

(306) transmitting parallel Information on at least one parallel chan- 
nel which parallel information is associated and synchronized with the media 
10 stream of at least one broadcast channel, 

(308) connecting the audience to the media system as passive us- 
ers or active users, the active users receiving the parallel information and the 
passive users not receiving the parallel information, 

(310) collecting information on the audience connected to the media 
15 system, the information being on active users and/or passive users, 

(312) processing the collected information in a predetermined way. 

2. The method of claim 1, characterized by the audience 
having an interactive connection to the media stream system via an interactive 
menu shown on the screen of a user terminal. 

20 3. The method of claim 1, characterized by collecting the in- 

formation on the audience in a memory. 

4. The method of claim 1, characterized by the collected in- 
formation being shown on the screen of a user terminal and/or a broadcaster's 
terminal. 

25 5. The method of any preceding claim, characterized by 

templates for the interactive menu or showing the collected information being 
ready-made, stored in a user terminal and being modified according to the 
parallel information. 

6. The method of any preceding claim, characterized by 
30 templates for the visual menu or for displaying of the compiled information be- 
ing transmitted as parallel information. 

7. The method of claim 1, characterized by collecting the in- 
formation on the active users by using an interactive menu on the screen of a 
user terminal comprising options to be selected and by using the parallel 

35 channel as a feedback channel for transmitting the user selections. 
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8. The method of claim 1. characterized by the parallel in- 
formation being transmitted as a packet transmission. 

9. The method of claim 1, characterized by collecting loca- 
tion information on the audience. 

5 10. The method of claim 1, characterized by using the col- 

lected information for searching for good times, geographical areas and/or 
modes for advertising. 

11. A media system for collecting information on the audience of a 
broadcast media stream, characterized by comprising: 

10 means (200) for broadcasting a media stream on at least one 

broadcast channel of the broadcast system, the broadcast system being part of 
the media system, 

means (104, 112, 202) for transmitting parallel information on at 
least one parallel channel which parallel information is associated and syn- 

15 chronized with the media stream of at least one broadcasting channel using a 
radio telecommunication system of the media system, 

means (104, 110, 112, 202) for connecting the audience to the me- 
dia system as passive users or active users, the active users receiving the 
parallel information and the passive users not receiving the parallel informa- 

20 tion, 

means (104, 110, 112, 202) for collecting information on the audi- 
ence connected to the media system, the information relating to active users 
and/or passive users, 

means (112, 202) for processing the collected information in a pre- 
25 determined way. 

12. The system of claim 11, characterized by further com- 
prising an interactive menu shown on the screen of a user terminal (110) for 
connecting the audience via an interactive connection to the media stream sys- 
tem, 

30 13. The media system of claim 11, characterized by further 

comprising means (420) for collecting the information on the audience in a 
memory. 

14. The media system of claim 11. characterized by further 
comprising means (104, 1 10, 1 12. 200. 202) for showing the collected informa- 
35 tion on the screen of a user terminal and/or of a broadcaster's terminal. 
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15. The media system of claim 11. characterized by further 
comprising means (104, 110, 112, 200, 202) for storing ready-made templates 
for the interactive menu or showing the collected infomiation and modifying 
them according to the parallel information. 
5 16. The media system of claim 11, characterized by further 

comprising means (104, 112, 200, 202) for transmitting the templates for the 
visual menu or showing the collected information as parallel information. 

17. The media system of claim 11, characterized by further 
comprising means (104, 110, 112. 200, 202) for collecting information on ac- 

10 tive users by using an interactive menu on the screen of a user terminal com- 
prising options to be selected and by using the parallel channel as a feedback 
channel for transmitting the user selections. 

18. The media system of claim 11, characterized by further 
comprising means (104, 110, 112, 200. 202) for transmitting the parallel infor- 

15 mation as a packet transmission. 

1 9. The media system of claim 11 characterized by further 
comprising means (104, 110, 112) for collecting location infomiation on the 
audience. 

20. A user terminal for collecting information on an audience of a 
20 media stream broadcast by a media system characterized by compris- 
ing: 

means (500. 506, 508, 514. 516, 520, 522, 524, 526, 528) for re- 
ceiving parallel information on at least one parallel channel which parallel in- 
formation is associated and synchronized with the media stream of at least one 
25 broadcast channel of the broadcast system, the broadcast system being a part 
of the media system, 

means (500. 502, 504, 516, 520, 522, 524, 526) for connecting the 
audience to the media system as passive users or active users, the active us- 
ers receiving the parallel information and the passive users not receiving the 
30 parallel information, 

means (500. 502, 504, 516, 520, 522, 524. 526) for changing a par- 
allel information reception state according to the user's selections, 

means (500, 502, 504, 516, 520. 522. 524. 526. 528) for transmit- 
ting selections made by active users for collecting information on the audience 
35 connected to the media system, 
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means (500, 506. 508. 514, 516. 520. 522. 524, 526, 528) for show- 
ing the collected infomiation. 

21 . The user temilnal of claim 20, characterized by further 
comprising means (520) for the audience to have an interactive connection to 

5 the media stream system via an interactive menu. 

22. The user tenminal of claim 20, characterized by further 
comprising means (520) for active users to make their selections by using a 
virtual menu on the screen of a user terminal comprising options to be se- 
lected. 

10 23. The user terminal of claim 20, characterized by further 

comprising means (500, 502, 504, 506. 508, 516, 520, 522, 524, 526) for 
transmitting and receiving parallel information as a packet transmission. 

24. The user terminal of claim 20, characterized by further 
comprising means (500, 502, 504, 506, 508, 516, 520, 522, 524, 526) for col- 

15 looting information on active users by using an interactive menu on the screen 
of a user terminal comprising options to be selected and by using the parallel 
channel for transmitting the user selections. 
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date and not in conflia with the applicadon but dttd to understand 
the principle or flieory underlying the invention 

'X' document or particular relevance: the daimed invention cannot be 
considered novd or cannot be conadcred to involve an inventive 
step vihm the document is taken alone 

" Y' document oT particular relevance: the daimed inventicn cannot be 
considered to involve an inventive step when the document is 
combined with one or more other such documents, such combinatian 
being obvious to a person slolled in the an 

'ft* document member of the same patent family 


Date of the actual con)pletion of the international search 
10 Julv 2003 


Date of mailing of the international search report 

1 5 -07- 2003 


Name and mmling address of the ISA/ 
Swedish Patent Office 
Box 5055, S-102 42 STOCKHOLM 
Facsimile No. +46 8 666 02 86 


Authorized oiftcer 

Stefan Hansson/mj 

Telephone No. + 46 8 782 25 00 
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date 
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AU 


6810701 
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EP 1096743 Al 02/05/01 
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Chef de la section "ofBce recepteur du PCT' 



PCX 



REQUEST 



The tmdCTigned re^$ts that the piesent 
accordiufi to tibe ffOKStX Cooperadan Treaty. 



Foir leceivlng OfHco ti$e only 



PCT / 1B 0 2 / 0 25 3 7" 

Iftlemational Application No. 



INTERNATIONAL BUREAU OF WlPO 

y«»e of iKa^AB?@ffl§l'ag^bMl^Rnal Application" 



Sffl^V^^^ttotOTw^S^ 990001 04/CHE 



Box No. 1 TITLE OF INVENTION 

A system and method for delivering representatK 


re media objects of a broadcast media stream to 


Box No. n APPLICANT 1%!$ person is also inventor 


Name and address'. (Famifyname/bllow^byg^imnnanw^fora'iegale^^ 

The adcfress must inshak pos(vl code and name eotmiiy, tws cauAy <^ the address imffcated in tfds 

Boxbthe^ppUcant's Siatefdmtls, eomby) tfrcsideticef^Sfat9^t^€9ldeneeirMiG»i«dbet^ 

NOKIA CORPORATION 
Kellalahdenfie4 
FIN-02150Espoo 
Finland 


Telephone No. 


Facsimile No« 


TeleprixmNo. 


App1icant'sx^$tmtionT4o.infl> theOfiSca 


state (thai is, country) of nationalit)r: 

Finland 


Stale ((has is, counlry) of residence: 

Finland 


This person is sqpplicawi rn «in<M8i»i«d aU desxawted Stata tfcoe^t 1 — IteUDittedBtates 1 — l&o States indicted in 
for tifc purpos»« 1 1 Slsira^ LcJ the ifflW Stews of Ammca I I of America only | 1 the Supplemental Box 


BoxNo. OT FURTHER APPLICANT® AND/OR (FURTHER) INVENTOR(S) 


Name and address: (iFdmtfj^navfe^&ffoNv^ 

The Gddfmnmtimh^epastal code atidnamt^CQiottry, Thzow«hyqf^addr€ss m^al&im UUs 
Box is the apphcani *s St0e (that is, <xnmtiy)qfrememeff»Staiei^reiitu^ itin^saSadbeluw,) 

MAKIPM. MIkko 
Airoranta 9 A 
FIN-00830 Helsinki 
Finland 


This person is: 
1 1 ^ppUcantenty 

1X1 applicant and inventor 

1 — 1 inventor only /to c^ecfc^ 
isnuxi'kedtdonotfillinbelQw.) 


AppUoaQt*8rB8istratiottNo.withttieOmce 


Stale t^at is, wunisy) of nationaUty: 

Finland 


State ^hat cr. comfy) of residence: 

Finland 


This person is applicant r~| all desigoated n aU designated States ex^ccpt RTl the United States | 1 to States Indicated an 
fbrthepmpoBe8oi& LJ Stales 1 I tbe Un£d States of Amo^ L£J of America on^y the Stqiiilatneetal Box 


Further i^Hcants and/or (fifftherjUltventDiisBxei^ 


Box No. IV AGENT OR COMMON REPRESJENTATIVi:; OR ADDRESS FOR CORRESPONDENCE 


The person idoQtilledbebw is herd)y/has been appoints 171 AMnt r^oonoiinon 
of tibie 2qppUcant(8) before tiw txni^e&at fot^m^liooal AuttKwrWes as: L£J 1 — 1 x^resentative 


Name and address: (Bamifyrum&foUowedhy^fivenfim^iforalq^ 

the addarm must include postal cock and na»ie of eoutttry^ 

AWAPATENTA/S 
TegiholmAlle13 
DK-2450 K0benhavn SV 
Denmark 


Telephone No. 

+45 70 20 00 33 


FacshnileNo. 

+4S 70 20 04 33 


TdeprintarNo. 


Agent's regisfradonNo. -with the Office 


Address for correspondence: Mark this checJk-box where no agent or common representative is/has been appointed and the 
LJ space above is used instead to hidioate a special address to whicb cottespon^tencB should be seat 
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PCX 

REQUEST 

The iindersigned requests that the present 

international application be processed 
according to the Patent Cooperation Treaty, 


PCT / IB 0 2 / 0 25 37 

International Application No. 


1 JULY 2002 ,/ n 1 

International Filing Date kC U I 07.02 ) 


INTERWATIOMAL BUREAU OF WlPO 

Name of recdHIg Office^ Application** 


1 Applicant's or agent's file reference ^ ^ 

1 (^desired) (12 dtaraders maximum) 990001 04/CHE 


Box No. I TITLE OF INVENTION 

A system and method for delivering representative media objects of a broadcast media stream to 


Box No. n APPLICANT |_| This person is also inventor 


Boxisthetq!pliamt*s^ate(ihat1s,caimiry)ofre5idenceifw 

NOKIA CORPORATION 
Kellalahdentie 4 
FIN-02150 Espoo 
Finland 


Telephone No. 


Facsimile No. 


Teleprinter No. 


Applicant's registration No. with the OfHce 


Mate (thai ts, country) of nationality: 

Finland 


State (that is, country) of residence: 

Finland 


&eTumosi^^Sf n siteT^"^ fXl Sll nS^H"*li?'*'f A^^* Fl United Stales |— 1 the States indicated in 
ror me purposes of. l f Slates L^H the Uniled States of Aroenca | | of America only 1 1 the Supplemental Box 


Box No. ra FmiTHER APPLICANT(S) AND/OR (FURTHER) INVENTOR(S) 


wS^Si? (FamUyncmiefoUoy^edbygivenname;foraleg^ 

■nte aaOvss must tnciude postal code and name of country, Tfie country of the address indicated in this 
Boxtstheqpplicani s State(thatis, coutt^)ofreskiencelfnoState<tfr&£nceisineHcatedbe!ow.) 

MAKIPAA, Mlkko 
Afroranta 9 A 
FIN-00830 Helsinki 
Finland 


This person is: 

1 1 applicant only 

|X| applicant and inventor 

1 1 inventor only (If this check-box 
1 — 1 is marked, do not fill in below) 


Applicant's registrationNo. with the Office 


Jilale (t/tat ts, country) of nationality: 

Finland 


State (titat is, country) of residence: 

Finland 




[Xj Further applicants and/or (furthw) inventors are Indicated on a continuation sheet 


Box No. IV AGENT OR COMMON REPRESENTATIVE; OR ADDRESS FOR CORRESPONDENCE 


The person identified below IS hereby/has been appointed to act on be^^ — . 

of the applicant(s) before the competent IntemauSJial Authorities as: S ^e^'** □ S^Suative 


Name and address: family namefolto\vedfy given name; for a legal entity JuUojf^^ 
The address must include postal code and name of country.) 

AWAPATFMT A/o 


Telephone No. 

+45 70 20 00 33 


Teglholm Alle 13 
DK-2450 Kobenhavn SV 
Denmark 




Facsimile No. 

+45 70 20 04 33 




Teleprinter No. 




Agent's registratlonNo. with the Office 


LJ ™ Marie this check-box where no agent or common representative is/has been appointed and the 
1 — 1 space above is used mstead to mdicate a special address to whicTi coirespondence should be sent «PPo»n»6a ana uie 



See Notes to the request form 



SUBSTITUTE SHEET (RULE 26) 



Sheet No. ...2... 



ContEnuatioa of Box No. IH FURTHER ArrUCANT(3) AND/OR (FURTHER) J[NVENTOR(5) 
If now of the following ^b^xes h used, this sheet should not be included in the request 


Name and syddress: (FamUynamefollowcdby given ivmejfi^rate^olml^ 

The addf^ss must include posted code tfxtd name of country. The country of uddres9 induced in this 

BoxisihtappHocoH*^ StateCthatb, country) ^reHdencel/HoS(ate(ifn^dknc€ Is tn£oa 

ANTTILA, Aksell 
Pajalahdentie 6 B 25 
FIN-00200 Helsinki 
Finland 


This DffirsoA is: 
1 i applicant only 

1x1 Gyppiioatuandinv^citor 

f~| inventor only (!{rftoQAafift^x 
1 — 1 is marked, AfUftJiU in bdtnf^^ 


Applicant'3regi$ti:aiiQnNo.withtheO£Eice 


State (that Is, country) of natioimU^ 

Finland 


State (that is, oomtty) of lesidenco: 

Finland 


Tlue person b appUcant l — i all designated | — | alt dcsJgoatcd States eji^cept the United States | — | the Stales indicated in 
for the puiposes of^ 1 1 States 1 1 &e United States of Amenca U&J of America only 1 1 tbe Supplemental Box 


Nam and address: (tamUynome/Qll<nvedby^vmft£metJbral^lm 
Tkead^bwrnstinduhpoMeodemidnume^eotailfy* viecoutlbytftheaM^inicatedinlhls 
BaxistheappUtmt*s State(ilienij, couniry)t^resSdeneefmSkfte^residBnc9 hMieatedhelow^ 

KOPRA, Ton! 
Saaritie 11 B 6 
FIN-03250 Ojakkala 
Finland 


Hiis person is: 

[ ^ appUcant only 

[X ] applicant and inventor 

1 1 inventor only (2jriWycftecfc-*ax 
l—i istnarked^danotfiUinbeloyf.) 


Applicant'8fegistralionNo.withfheO£fice 


State {that is» country) of nationality: 

Finland 


state (dtai is, country) of xesidexice: 

Finland 


This peraon is applicant i — | alldesignaled i — i an designated Stat^ excqpt flFl the United States | — 1 the Stales mdkated fo 
for Uio purposes of: LJ S'ts^ «e Umted Siatoof Anmoa Uy of Amedcaonly | j UxeSupplematfalBox 


Name and address: (Ftmilyrtctmefalhy^bygivmnamsijcrattg^ 

Vie addr^s must include postal code end name country. Vte cotmiry ofOte address indicot&iin this 
BoxistfteappticoFU's State Qkatis, cotihtty)o/reridenee^»oSta(etftesi&nceis(n(aeatedbdo^^^ 


Hiis person is: 

rn applicant only 

\ 1 applicant and inventor 

1 1 inventor only (^^Acr cAec^^ox 
t~J ismwioeidonotftllinbelowj 


Applioant'8regi5tiadonNo.withtfaeO£6ce 


State (Oua iSt oomty) of natioxxaUiy: 


state (AeU is, country) 


of residence: 


This peisoa is aj^cant | l aU designated l — 1 eU des Wed States exceot I — | the United States | — | the States indicated in 
for tfcke purposes of: 1 — 1 States j 1 the United Stkes of AmSSca 1 I of America on^r | I the Supplemental Box 


Name and address: (Family rtamefiflloYfedby^ivm name; for ale^lm^ 

Thz address must includepostal code and name qfcounny. The comtryitfihe address in£catedin this 

Boxisth9appiieant*sSiat*(ihatis,oowtry)t^resid 


This person is: 
1 1 ^iicantonly 

1 J applicant and inventor 

1 I .inventor only flf this cheM>ox 
1 — 1 ismarh^donotfiUinbekiyif,) 


At^plicanfaregistrBtionNo. with the Office 


State (that is, country) of nationally 


State (^t is, country) of residence: 


Tlliapereoo is applicant l — l ^dej^gnated 1 — | ^ desisted Staire exee^ l — i the United States 1 — i the States indicated in 
for the piacposes of: 1 1 States | | the Uniied States of Amroa 1 j of America only LJ Supplcanental Box 


[3 I'urther applicants and/or (fiinher) ioventon are indicated on another continuation sheet 
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Box No. V DESIGNATION OF STATES JS^arkthe<^pUcable<^eckyboxesl^U>^^^ 



The foUowix\g desigtutions are hereby n»de under Rnle 4.9(a): 
Regional Patent 

B AP ARIPO Patent: GH Ghana. GM Gambia, KE Kenya, W Lesotho, MW Malawi. MZ Mozambique, SD Sudan, 
S)L Sierra Leone, SZ Swaziland, TZ United Republic ofTanzanxa, UG Uganda. ZMZambia^ZW Zimbabwe, and any other 
State v/hicsh is a Contracting State of ^ Harare Protocol and of the PCT (J/oift»* Idnd ofproiectim or treatment desir^ 
specify on dotted line) » » » • * . . . - 

B EA ^;nra8ianPatBnt:AMAnn6ttla.AZAzerba«an,BYBelaru5,KGKyrsy2$tan^ 

RU Russian Federation, TJ T^lkfctan, TM Turkmenistan, and any other State which is aConlracting State of the Bwasxaa 
Patent Convention and of the PCT 

B EP Euit^peanPatcnt: ATAu5tria,BEB6lgiura,BGBulgaria,CH&LI5witzerla»dandUechtenste^ 

Kj^ublic, PE Genneuy, PK Denmark, EE Estonia, ES Spain, FI Finland, FR France, GB United Kingdom, GR Greece, 
IE Iieland, IT Italy, LU Luxembourg, MC Monaco, NL Netherlands, PT Portugal, SE Sweden, SK Slovajda, TR Turkqr, and 
sny other State which is a Contracting State of the European Patcait Convention and of the PCT 

B OA OAPI Patent: BP BuridnaPaso, BJT Benhi, CP Centtal Aftioan R^ubUc, CG Coogo, CI Cote d'lvoire, CM Cameroon, 
GA Gabon, GN Guinea, GQ Eqaatorial Guinea, GW Guinea-Bbsau, Ml^ Mali, MR Mauritaiua, NE Niger, SN Senegal, 
TD Chad, TO Togo, and any other State which is a member State of O API and a Contracting Stale of the PCT fjf other Idnd 
of^tecdvn tjr treatment desired, spedtfy on dotted Hne) • 

National Patent (pother kffidofprotection or treatment desireii^pecffif on dotted 

B AE United Arab Emiiates B GMGarabia » NZ New Zealand 

B AG Antigua and Baibuda H HR Croatia B OMOman 

B AL Albania B HU Hungary B PH PhiMppines 

B AMAnaenia B ID Indonesia B PL Poland 

B AT Austria B IL Israel B PT Portugal 

B AU Australia B IN India B RO Romaida 

B AZ Azeibajyan B IS Iceland B RU Russian Federation 

B BA Bosnia and HeraegovJna* B JP Japan 

Bbb Barbados BKEKeqyB.,. B SD Sudan 

B BG Bulgaria B KG Kyrgyastan H 5£ Sweden 

B BR Brazil B KP Democratic People's R^ublio B SG Singapore 

B BY Belarus of Korea B SI Slovenia * 

B BZ Belize B KR RqpubUc of Korea B SK Slovakia 

B CA Canada B KZ Kazakhstan B SL SierraLeone 

B CH&XJ Switzerland and Lieditenstein B LC Saint Luda B TJ Tayikisian 

BCN China B LK Sri Lanka B IM Tmkmenistan 

B CO Colombia B LR Liberia B TN Tunisia 

B CR Costa Rica B JLS Lesotho...- B TO Turk^ 

B CU Ciiba DB LT Lithuania B TT Trinidad and Tobago 

B CZ Czech Republic B LU Luxembourg 

B DE Gennaty B LV Latvia B TZ United Republic of Taroania 

BpK Denmark B MA Monocco B UA Ukraine 

B DMDominica B MP RepubUo of Moldova B UG Uganda 



B US United States of America , 



B UZ U^Mdatan. 
B VN Viet Nam.. 
B YU Yugoslavia 



B I>Z Algeria 

B EC Ecuador B MG Madagascar 

B EE Estonia B MKThe fonn^ Yugoslav Ri^ublio of 

B ES Spain Macedonia .«•»..« 

B FI Finland B MNMongoUa 

B GB United KitJgdom B MWMalawi B ZA South Africa* . . - 

B GD Grenada B MXMexico B ZMZamibla 

B GE Georgia B MZ Moaanibiqua H ZWZimbabwe 

B GR Ghana B NO Norway 

Ched(-boxes below reserved for designating States which iiave become party to the PCT alto issuance of this sheet: 

□ □ □ 

□ □ □ 



Precautionary Designation Statementt In addition to the designations made above, the applicant also makes under Rule 4.9(b) all 
other designations which would be permitted under the PCT except any de5ignalion(s) mdicated in the Supplemental Box as being 
excluded from the scope of tiiis statement The g^pticant declares that those additional designations are subject to confizmation and that 
any designation which is not confirmed before the expiration of 1 5 months itom the priority dale is to be regarded as witfxdrawn by the 
^Ucantatdxeexpiraiionofthattimeiiimt. (Cot0rmation (imdueBng/ees) mustraxchtheteeetvtngOfficeMHthitt the iS-nwnth time WnU.) 
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Box No. VI PRIORITY CLAIM 



The priority of the foJlowing earlier applicalio0(s) is hereby claimed: 



Filing date 
of earlier appHcation 
(day/mon^ifyear) 



item(l) 



item (2) 



Number 
of earlier fi^Ucation 



Where eailier application is: 



ttadoDEl application: 
country 



regional appUcailon:* 
regional Office 



iotexnatio&al opplication: 
reoeivrogOgice 



item (3) 



item (4) 



ftem(5) 



P I^urtliwpriMilyolalmsaxtt indicated in the Supptement^ 



The receiving OfiEice is requested to prepare and transmit to the International Bureau a certified copy of the earlier aCTlication(5) (only 
If Ike eartkrc^UcaiiQn was filed wUh the Office v/Mch^r the purposes <^(his iniemaiional application Is the receiving CffvGe) identified 
above as: 

□ ailitems □ item(l) □ item (2) □ item (3) □ item(4) □ item(5) □ l^^l^talBo^ 

^ Where (he earikr explication is an ARtPO appHcation, indicate at least one country parfy to the Paris Cometaionjbr thePnotecfu^ of 
Irtdustrta! Property or one Member ^ the WorU Trade Organkation/or which that earlter tqjpiicaihn was JUed (Rule 4JQ(b}0$): 



BozNo.Vn INTERNATIONAL SEARCHING AUTHORITY 



Choice of Internation^ Searching Authority (ISA) Of two or more Intematloned Searching Authorities are competeni to carry out the 
mtomational search, indicate the Authority chosen; the two-letter code nu^ be usee^: 

ISA/.^P 

Request to nae reauits of earBcr seardi; refenatee to that search (if an earlier search has been carried out by or requested from the 
fnternationcd Searching Au^Hty): 

Date (dqy/month^ar) Number Country (or re^onal Office) 



BoxNo.Vm PECLARATIONS 



The following declarations are contained in Boxes Nos« Vin (i) to (v) (mark the epplicoMe 
cheeh4>oxe9 bdaw and indicate in rigfU coham the number of each type ofdedLaroAorOx 



Number of 
deolataiions 



□ BoxNo.Vin(l) 
Bo9cNo.vm(ii) 

□ BoxNo.Vin(iii) 
jjg BoxNo.Vni(iv) 

□ BoxNo.Vm (V) 



Declaration as to the identity of the inventor 

Dedaration aa to the appUcant*a entitlement, aa at lite intemattonal fiUng 
daie» to i^ply for and be granted a patent 

Declaraiipn as to the applicant's entitlement, as at the international filing 
date, to claim the priority of the earli^ application 

Peclaration of inventorship (only for tiie purposes of the designation of the 
Uuited States of Amenca) 

Declajcation as to non-prejudicial discbaurea or exceptioxu to lack of novelty 



Form PCT/RO/1 01 (Aird Sheet) (March 2001; reprint January 2002) 
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Box No. vm (u) DECLAJRATION: ENTITLEMENT TO APPLY FOK AND BE GRANTED A PATENT 
Thedeclmthnmt^cofifQrmfothestandardizedyM see No^esU? Boxes Nas. VSl, V12T^ U>(v) flngen&xtQ 

and (he. specific Notes to Box, No. VUl ffitus Box is not used, this sheet ^hotdd not be included in the request 

Deolmt&QO as to the appUcaot's entitleiiaeow as at the international filmg datei to apply for and be granted a patenit ij^ts 4 J7(ii) 
and S]|»2sf.l(aXiO)r ^ wh^xe the declaration vmder Kule 4.17(iv) is col qpi«rqpfiaie; 

in relation to the present international patent application 

NOKIA CORPORATION is entitled to apply for and be granted a patent by virtue of the 
following: 

(11) NOKIA CORPORATION is entitled as employer of the Inventors, Mlldco Makipaa, 
Aksell Anttila and Toni Kopra. 

(Ix) this designation Is made for the purposes of: . 

(a) all designations [except the designation of the United States of America] 



[jl This dedaiation is eontmued on the followii^ oheet, *K::Qminuati[on of Box No. VM QJ)?, 
Fonn PCT/BO/1 01 (deolsxaUon 9beet (11)) (Mar^ 2001 ; reprint Januegry 2002) See Notes to the request fiym 
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Box Nq. vm (iv) PECXAKATION: INVENTORSHIP (pnly fcrtliBpuiposes of tfee draignatigii of the United States of America) 
Thed^ara&mmmconfQrmtothefottowb^su^ see Notes to B^xesNos, Vm, VmO)t9(v) 

(in generaO <md the ^cffh Ndtes to Box No. VBl OvJ. IfthisBoxis not used, m sheet should mt be inobtded in the request. 



PecUration Qf InventorsUp (Rules 4.17Qv) and 
for the purpose of the designation of the Untied States of America: 

I hwljy declare that I believe I am ihe ori^al, fiist and sole (If only one inventor is listed below) or joint (if moie than one invrator 

i§ listed below) inventor of the subject matter which is claimed and for which a patent is sought. 

This declaration is directed to the international application of which U foims a part Cif filing declaration with application). 

This declaration is directed to international application Wo, PCT/ (If f unushing declaration pursuant 

to Rule 26/er). 

I hereby declare that my residence, mailing address, and citizenship are as stated next to my name. 

I hercl>y state thai I have reviewed and understand the contents of the abovc-identiJfied international application, including the claims 
of said application. I have identified in the request of said appUcadon, in compliance with PCT Rule 4.1 0, any claim to foreign piionty, 
and I have identified below, under the heading 'Trior Applications," by application number, countty or Member of the World Trade 
Organisation, day, month and year of filing, any application for a patent or inventor's certificate filed in a country other than theUmtcxi 
Stateaof America, intruding atiy PCT intemationalappUcationdBSigttathxg at least 0 
having a filing date before ^ of the application on whic^ fbrelgn priority is clahned. 

Prior Applications: ' 

1 hereby acknowledge the duty to disclose infbtttiatlon tt»t is known fey me to be material to patentabiUty as defmed by 

37C.F.R § J.56,includii^8 for cottfeuaiion-in-ipaTtappUcations, material infbrmad^ 
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A SYSTEM AND METHOD FOR DBLIVBRINQ RBPRESEMTA -TIVB MEDIA, 
OBJECTS OF A BROftPCAST MEDIA STREiVW tq a TERMI1«M. 



Field of Invention 

This invention relates to a system and method for delivering 
media objects associated with a broadcasted media stream to one 
or more terminals. 

Background of invention 

Media streams such as a radio or television transmissions, 
videos, or DVDs are generally controlled and presented by a 
communication system such as a radio or television receiver, a 
video recorder or DVD player. A user may control the 
presentation of the media stream by directly operating the 
communication device eo as to select a particular media stream 
or particular sequence of the particular media stream. The 
Jcnown media streams and known communication devices are 
satisfactory in the presentation of a media stream in a 
sequential fashion. However, when it comes to commercial 
utilisation of the media stream the known communication systems 
may be further improved in order to serve multi-technical 
communication systems, i.e. combinations of communication 
systems, and to fulfil future customer demands for versatility 
of their communication devices and media stream products. 

Summary of the invention 
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An object of the present invention is to provide a system and 
method for providing media objects of a media stream^ which 
media objects are created for and made available to a user of a 
communication system during a primary media stream experience. 

5 

A further object of the present invention is to capture a frame 
of a media stream of a broadcast or video transmission to a 
terminal . 

10 A particular advantage of the present invention is provision of 
editing capability for editing a frame of a media stream so as 
to personalise the frame- 

A particular feature of the present invention, relates to the 
15 provision of a "^capture of the moment" or ^^record" hot key in a 
terminal enabling a user to download a frame from a plurality 
of frames of a media stream. 

The above objects, advantage and feature together with numerous 
further objects, advantages and features, which will be evident 
from below detailed description, is accomplished by a solution 
in accordance to a first aspect of the present invention by a 
system for delivering a media object to one or more terminals, 
which media object is associated with a media stream 
broadcasted to one or more media stream receivers, said system 
comprising: 

(a) a broadcasting network for connecting to said one or more 
media stream receivers; 

(b) a first communications network for connecting to said one 
or more terminals; and 

(c) a communication device connecting to said broadcasting 
network and broadcasting said media stream to said one or 
more media stream receivers, and connecting to said 
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communications network and cotnmtjni eating said media object 
to ©aid one or more terminals. 

The term ^one or more'' should in this context be construed as 
a, an, at least one, at least a single. 

The media stream according to the first aspect of the present 
invention may comprise a television and/or radio transmitted 
show, drama, movie, sports game, news, or any combination 
thereof- Thus any type of television emd/or radio transmission 
may constitute a media stream. 

In addition/ the media object may comprise a text, a picture, a 
series of pictures, a video, a series of videos, an audio 
15 recording, a series of audio recordings ^ or any combination 
thereof. Hence the media object may comprise any related or 
unrelated information in regards to the media stream and may be 
presented in any type of readable format. 

The termixial according to the first aspect of the present 
invention may comprise a phone, a cellular or mobile phone, a 
personal computer, a television, a set top box, a multimedia 
terminal, a personal office assistant or any combination 
thereof, and the one or more media stream receivers may 
comprise a set top box, multimedia terminal, television 
receiver, television, radio receiver or any combination 
thereof • 

The communication device according to the first aspect of the 
30 present invention may broadcast to the one or more media stream 
receivers by a cable television network, a satellite television 
network, a radio frequency television network, a telephone 
network, a powerline network, a radio network or any 
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combination thereof. Thus any type of network may generally be 
applied for broadcasting of the media stream, i.e. various 
types of providers capable of transmitting the media stream to 
the receivers may be used. This provides for a system, which is 
very versatile. 

The communication device according to the first aspect of the 
present invention may be adapted to transmit digitally coded 
communication such as digital video broadcasting and/or digital 
audio broadcasting. The digitally coded signals provide better 
transmission quality and enables the comraunication device to 
forward additional information to a receiver. 

The first communications network according to the first aspect 
of the present invention may comprise a telephone wire network, 
a cable television network, a powerline network, a computer 
network, a wireless telephone network, or any combination 
thereof. The communication between the communication device and 
the one or more terminals may utilise a wide variety of network 
types and wide variety of combinations of network types 
depending upon which provider is selected. 

The communication device according to the first aspect of the 
present invention may comprise a broadcasting unit for 
broadcasting the media stream to the one or more media stream 
receivers, a management unit for providing the media object to 
the one or more terminals, and a second communications network 
for interconnecting the broadcasting unit and the management 
xinit. The broadcasting unit may comprise a marker for 
generating a media stream identification tag associated to the 
media stream, which media stream identification tag may 
comprise information regarding duration of the media stream, 
lapsed time of the media stream, broadcasting schedule for the 
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media stream, broadcasting channel for the media steam, or any 
combination thereof. By tagging the media stream with an 
identification mark any specific media objects relating to the 
media stream may be connected to the media stream in the 
5 communication device bo as to provide a tool for managing the 
media objects. 

Further, the broadcasting unit may be adapted to perform a 
continuous communication of data regarding the media stream 
10 information tag, an updating communication of revision of 
specific data regarding the media stream information tag, a 
communication based on schedule of the media stream, or any 
combination thereof. Any of the above reference commoni cat ions 
are advantageous since they all serve a specific purpose. 
15 Continuous coraraonication enables the broadcast unit to 

continuously correct for changes in the broadcast scheduling of 
the media stream and to continuously create new media objects 
relating to the media stream. Updating communication similarly 
provides the broadcasting unit with the possibility to adjust 
2 0 for changes in the broadcast scheduling of the media stream. 
Finally, the scheduled communication such as predefining 
intervals in which the broadcast unit may communicate with the 
management unit provides a well structured and coordinated 
communication form. 

25 

The management unit according to the first aspect of the 
present invention may comprise an application program Interface 
for receiving the media object, a database for storing the 
media object and the media stream identification tag, a real 
30 time publishing interface for enabling real time publishing of 
the media object, and a user interface for providing the one or 
more terminals access to select the media object stored in the 
database through the first communications network. The various 
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interfaces may be inrpletnented in a plurality of format so as to 
support a wide range of coniniunication standards. 

The user interface may be adapted to respond to a request from 
5 the one or more terminals and to generate a media object list 
of media objects^ which are accessible for the one or more 
terminals , 

The second communications network according to the first aspect 
10 of the present invention may comprise local area network, 

metropolitan area network; wide area network, or inter-network 
such as the Internet, a dedicated communication line, or any 
combination thereof. The first aspect of the present invention 
may therefore be implemented for any particular network being 
15 wireless or hardwire. 

The system according to the first aspect of the present 
invention may further comprise a billing unit for managing 
billing transactions for the one or more terminals' recpiests 
20 for the media pbject and for generating invoices to the one or 
more terminals in accordance with the billing transactions- The 
transactions may be recorded so as to present invoices to users 
of the system. 

25 In addition, the system may further comprise a third 

communications network for interconnecting the billing unit and 
the management unit and a fourth communications network for 
interconnecting the billing unit and the one or more terminals. 
The third and fourth communications network conrpriaes local 

30 area network, metropolitan area network, wide area network, or 
inter-network such as the Internet, a dedicated communication 
line, a telephone wire network, a cable television network, a 
powerline network, a computer network, a wireless telephone 
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network/ or any combination thereof. As described above the 
network type is not limited since the system may be 
incorporating into any known network types, 

5 The term broadcasting network is to be construed as a cable TV 
network/ a satellite TV network, a radio frequency TV network, 
a radio cable or terrestrial network, and/or any TV or radio 
network utilising digital transmission techniques - 

10 The communication device according to first aspect of the 

present invention may broadcast the media stream and the media 
object through the broadcasting network and the one or more 
media stream receivers may connect to the first communications 
network and communicate the media object to the one or more 

15 terminals. The media object may be broadcast through a digital 
television network as part of the media stream such as through 
super text TV. A digital receiver such as a set -top box may 
store the media objects and communicate them subsequently to 
the one or more terminals. 

20 

The above objects/ advantages and features together with 
numerous further objects, advsuitages and features which will be 
evident from below detailed description is accomplished by a 
solution in accordance to a second aspect of the present 
25 invention by a method for delivering a media object to one or 
more terminals, which media object is associated with a media 
stream broadcasted to one or more media stream receivers, and 
said method comprising: 

(a) associating said media object. with said media stream by 
30 means of a communication device; 

(b) broadcasting said media stream to said one or more media 
stream receivers through a broadcasting network by means 
of said communication device; and 
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(c) commxmicating eaid tnedia object to a requesting terminal 
of said one or more terminals through a communications 
network by means, of said conimunication device. 

The method according to the second aspect of the present 
invention may further comprise defining a parameter for the 
media object by means of the communication device and the 
parameter defining a media object format such as audio, video, 
image, or any combination thereof, a technical format, an 
alternative task such as full view or close-up, a terminal 
requirement, or any combination thereof. By defining the 
parameter the method provides for an effective means for 
selecting those media objects which are readable to a specific 
terminal . 

The method according to the second aspect of the present 
invention may further comprise packaging a set of media objects 
associated with the media stream and publishing the set of 
media objects to the one or more terminals by means of the 
communication device. The packaging may comprise linking the 
media object to the media stream so that the media object is 
attached to a broadcasting time line of the media stream and 
defining the availability of the media object in accordance 
with the broadcasting time line of the media stream. Obviously, 
some coxmection between the media stream and the media objects 
is required in order to manage a media object relative to a 
terminal and relative to a media stream. 

The packaging may further comprise defining a media object 
based on a key moment of the media stream as an elapsed time 
from the start of the media stream, defining the media object's 
availability prior, during and after broadcast of the media 
stream, defining an additional time period during which the 
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media object's availability is announced but not available for 
transfer, or any combination thereof. Any desired part of a 
media stream may be utilised for the creation of a media object 
associated with the media stream. 

5 

The method according to the second aspect of the present 
invention may further comprise managing the set of media 
objects by means of the communication device^ and the managing 
comprises controlling availability of each media object of the 
10 set of media objects in accordance with the broadcast time line 
for the media stream. The availability of the media object is 
controlled so as to provide a constant high level of current 
interest in the media objects. This motivates a user to further 
use the method for downloading more media objects. 

15 

The method according to the second aspect of the present 
invention may further comprise providing the one or more 
terminals access to the availedDle media objects and enabling a 
requesting terminal of the one or more terminals to transfer 

20 any specific available media object. The providing may comprise 
presenting a user interface to the one or more terminals/ which 
user interface lists the set of media objects. The user of a 
terminal may thus select from a set of media objects associated 
with any particular media stream. The number of media objects 

25 in a set may vary in accordance with the popularity of the 
media stream. 

The method according to the second aspect of the present 
invention may further coit5>rise generating a media object by 
30 means of the commxinication device in response to a recjuest from 
the one or more terminals. The request is accomplished by a 
user of a terminal depressing a hotkey for capturing a key 
moment of the media stream. The user Interface presents a 

la 
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specific list for a specific terminal, which specific list 
coinprises a media object/ which is readable by the specific 
terminal. The user of a terminal may thus by depressing a 
button on his terminal initiate the creation of a media object 
5 to a media screen. In this way the user may select any frame or 
sound he desires from the media stream. 

The method according to the second aspect of the present 
invention may further comprise purchasing the media object from 
10 the communication device by means of the one or more terminals, 
by purchasing the media object the media object is transferred 
to the one or more terminals. Since media streams may be 
subject to royalties the user of the method should be at least 
self supporting or part of a business « 

IS 

The method according to the second aspect of the present 
invention may further comprise recording and processing of the 
transfer of the media object to the one or more terminals by 
means of a transaction processing device. Payment of the 
20 utilised services may be monitored in a wide variety of ways 
thus the method opens the possibility for itt5>lementation in 
many circumstances. 

The method according to the second aspect of the present 
25 invention may further comprise identifying the media object 

format by means of the one or more terminals, the identifying 
revealing information such as supporting application needed, 
additional rights pertaining to the media object, forwarding 
limitations associated with the media object, or any 
30 combination thereof* 

The method according to the second aspect of the present ' 
invention may further comprise providing privileges associated 
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with the media object and with the one or more terminals. The 
privileges enable the one or more terminals to copy or forward 
the media object in accordance with each of the one or more 
terminals' number of purchases of said media object. In 
5 addition or alternatively, the may privileges disable the one 
or more tearminals to copy or forward said media object. Further 
in addition or alternatively, the privileges may disable the 
one or more terminals to copy or forward more than the one or 
more terminals' number of purchases* It is important to monitor 

10 and restrict the user of the terminals in exploiting the media 
objects beyond the rights pertained thereto. Hence the method 
according to the second aspect of the present invention may 
ensure against this type of exploitation. The privileges may be 
incoarporated in the system according to the first aspect of the 

15 present invention. 

The method according to the second aspect of the present 
invention may further incorporate any features of the system 
according to the first aspect of the present invention. 

20 

Brief description of the drawings 

The above, as well as additional objects, features and 
advantages of the present invention, will be better understood 
25 through the following illustrative and non- limiting detailed 
description of preferred embodiments of the present invention, 
with reference to the appended drawings , wherein s 

Figure 1 shows a system according to a first embodiment of the 
30 present invention; 

Figure 2 shows an example of the methodology used by the system 
according to the first embodiment of the present invention; 
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Figure 3 shows an overall view of key coirponents of the system 
according to the first ethbodinient of the present invention; and 

Figure 4 shows a flow chart of niethod according to a second 
embodiment of the present invention. 

Detailed descript ion of pref ^r^^ ^^ odiments 

In the following description of the various embodiments, 
reference is made to the accompanying drawings which form a 
part hereof, and in which is shown by way of illustration 
various embodiments in which the invention may be practiced. 
It is to be understood that other embodiments may be utilized 
and structural and functional modifications may be made without 
departing from the scope of the present invention. 

A communication system according to a first embodiment of the 
present invention is shown in figture 1 as designated in its 
entirety by reference nximeral 10. 

The communication system 10 enables a user of a terminal 12, 
such as a cell or mobile phone, during a media stream broadcast 
to capture a media object. A media object should in this 
context be constarued as a frame of a media stream, a series of 
frames of a media stream, a video sequence of a media stream, a 
part of a sound track of a media stream, or any combination 
thereof . 

The communication system 10 further comprises a display 14 for 
displaying a broadcasted media stream 16. The display 14 is 
communicating with a receiver 18, such aa an external or 
internal digital set -top box, a digital receiver, or an 
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analogue receiver. The receiver 18 may in an alternative 
eitibodiment of the present invention fxirther coitiraunicate with a 
video recorder, a DVD player, a radio receiver, a sovind 
amplifier, or any combination thereof. 

The media stream IS is broadcasted through a broadcasting 
network such as a cable television network, a satellite 
television network, a terrestrial television network, a 
telephone network, a powerline network, a cable or terrestrial 
radio network or any combination thereof. 

Tjie terminal 12 may comprise a hot key 20 enabling a user of 
the terminal 12 to select, by depressing the hot key 20, a 
media object 24 associated with the media stream xs. Thus, a 
user of the terminal 12 watching a television show may during 
the show depress the hot key 20 and thereby request a media 
object to be transferred through a communications network, such 
as a wired or wireless telecommunication network. In addition, 
the user of the terminal 12 may select any part of the media 
stream 16 such as any particular audio tracks from the media 
stream 16. 

The communication system 10 enables turning existing mass media 
properties into further digital merchandise by utilising the 
familiarity and appeal of characters, events and themes songs 
of particular television shows, movies or radio programs for 
media objects to be incorporated into a user's terminal 12. 

The media object 24 is created as a representation of a 
particular scene of a television show, however, the media 
object 24 may be any key moments of television shows, movies or 
radio programs such as high points of the plot line (Ross and 
Rachel's first kiss in the television series "friends*'), a 
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clever pimch line in a television show, a particular comment by 
a character in a movie or television show, a goal scored in any 
sports game- The media object 24 may be a video clip, a 
picture, a series of pictixrea, animations, soundtracks or the 
like. 

The media object 24 when transferred onto the terminal 12 may 
be used as any personal terifninal enhancement such as background 
images, ringing tones, messages, or logos. The terminal 12 
comprises an editor for enabling a user of the terminal 12 to 
edit the media object 24 in accordance with any personal 
preferences. The terminal 12 further comprises a memory for 
storing of the media object 24 so that the user may further 
coramuixicate the media object 24 per se or an edited version of 
the media object 24 to other terminals through a wireless 
telecommunications network and/or utilising a multimedia 
messaging service * 

An exanple of the methodology is shown in figure 2. A 
communication system designated in its entirety by reference 
nvimeral 40 comprises a television set 42 having a monitor 44 
and a receiver 46 and displaying a specific media stream 48, 
and comprises a communication device 47 for broadcasting the 
specific media stream to the receiver 46 and for providing 
media objects related to the specific media stream to any 
number of terminals. 

A first user is watching the media stream 48 on the television 
set 42 and desires to transfer a media object 50 associated 
with the media stream 48 onto a first terminal 52, which 
transfer is shown as an arrow 54. The first user captures this 
media object 50 by using the first terminal 52 and pressing the 
"capture the moment" hot key 56. The first user may then want 
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to share the media object SO with a second user of a second 
terminal 58 and hence in accordance with a set of privileges 
associated with the first user communicate the media object so 
by utilising for exan^jla a multimedia messaging service (MMS) , 
which communication is shown in figure 2 as an arrow 60. The 
second user of the second terminal 58 may subsequently be using 
the media object for example as a personal background on 
his/her terminal or in accordance with a set of privileges 
associated with the second user distribute it further by 
sending for example a new e-mail with the media object attached 
with the e-mail transmission. 



The utilisation of the multimedia messaging service for 
forwarding or sharing the media object may be incorporated in 
15 the terminal as a application program presenting a menu 

enabling the user to activate a transfer of the media object. 
In addition^ the application program may open a recipient 
window to be filled by the user of the terminal prior to 
activating a transfer of the media object and in this process 
offering the user the use of the address book of the terminal. 
Further, the application program may con^jrise a editing 
facility enabling the user of the terminal to edit in the media 
object prior to activating the transfer of the madia object. 
The editing facility the user may add comments to the media 
25 objects. Furthermore, the application program may enable the 

introduction of an advertisement to be associated euid forwarded 
with the media object. This renders it possible to have 
messages partly of fully financed by the advertiser who wants 
to sponsor the media object. 



Both the first and second terminal 52 and 58 provide the first 
and second user with the possibility to edit the media, object 
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50 SO as to personalize a raeasagl In conjunction with the 
transmission of the media object from one terminal to the next. 

Figure 3 shows an overall view of the key components of the 
> conmnmication system lo. The comtmmication system 10 comprises 
a broadcast unit 80 for broadcasting a media stream to one or 
more television receivers in any given region 82. The broadcast 
of the media stream is shown as a first arrow 84 and may be 
accomplished by wireless, cable or satellite transmiBsion. The 
region 82 may be defined by a cable television network or a 
plurality of individual television receivers. 

The broadcast unit 80 comprises a marker 86 for continuously 
generating an associated media stream identification tag to a 
specific media stream to be aired on a specific transmission 
date and time. The tag may contain further information 
regarding duration of the specific, media stream, lapsed time of 
the specific media stream and tr^mission channel. 

The broadcast unit 80 communicates the associated media stream 
identification tag to a management unit 88 through a first 
communications network 90 such as local area network, 
metropolitan area network, wide area network, or inter-network 
such as the Internet, or alternatively on a dedicated line. The 
management unit 88 comprises an application program interface 
92 for receiving one or more media objects 94 associated with - 
specific media stream identification tags and connects to a 
database 96 for storing the one or more media objects 94 
together with the associated media stream identification tag. 

The broadcast unit 80 may perform a continuous transmission of 
data regarding a media stream information tag, an updating 
transmission of revision of specific data regarding a media 
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stream information tag, or a transmission based on a scheduled 
program listing. 

Furthermore, the management unit 88 conprises a real time 
publishing interface 98 enabling real time publishing of media 
objects. That is, enabling publishing of the one or more media 
objects 94 during the broadcast unit's broadcast of an 
associated media stream. 

In addition, the management unit 88 comprises an interface 
processor 100 for providing one or more terminals 102 access to 
the one or more media objects 94 stored in the database 96 
through a wireless communications network 104. The interface 
processor 100 responds to a call from one or more of the 
terminals 102 and generates a media object list of the one or 
more media objects that are accessible for the specific one or 
more terminals 102 at that specific date and time. The access 
of the one or more media objects 94 may thus be limited to a 
specific time period so as to create media objects which are 
dependent on the transmission of a-medik stream. 

The management unit 88 further communicates with a billing 
device 10€ through a second communications network 108, which 
may be any of the above types mentioned with reference to the 
first communications network 90, it may in fact be the same 
communications network. The billing device 106 manages 
transactions of media objects, accounts of the one or more 
terminals 102, and generates invoices. The billing device 106 
may further communicate with the one or more terminals 102 
through a third communications network 110, which as before may 
be any of the above types mentioned with reference to the first 
and second communications network 90 and 108 in combination 
with a wireless communications network. 
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In an alternative embodiment of the raanagetaent unit 88 
cotnmiHiicates the tnedia objects through the second comnrunication 
network 90 to the broadcast imit 80/ which communicates the 
media objects to the one or more television receivers or set- 
top box in the region 82 together with the media stream. The 
media objects may be communicated as a super text TV object. 
The one or more television receivers communicate the media 
objects to the terminals through a wireless communications 
network 105 . 

Figure 4 shows a flow chart of a method according to a second 
embodiment of the present invention/ which method is designated 
in its entirety by reference numeral 120, The method 120 
comprises a start 122 for performing initialisation of the 
method 120. The start 122 involves establishing an interface 
for media object generating devices, such as the brx^adcast unit 
80 described with reference to figure 3/ and an interface for a 
terminal/ such as the one or more terminals 102 described with 
reference to figure 3, 

The media object generating devices utilise a management unit, 
such as described with reference to figure 3 as the management 
unit 88, for establishing tnedia objects 124 on or under the 
control of the management * "unit * The media object generating 
devices use an original media stream content for creating the 
media objects associated with said original media stream. The 
media objects may be key frames or key sounds, which define a 
special moment of the media stream. The media objects ar^ 
created prior to broadcasting the media stream, which 
broadcasting may be performed one or more times in one or more 
regions or in one ore more television network. However, the 
media objects may, in addition, be created during a broadcast 
of an associated media stream, for exatrple during live sports 
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broadcasts, which enable a terminal to select representative 
frames of the media streata (sports broadcast) such as a goal 
scored during a soccer match. 

5 The media objects may be implemented in any format such as in 
Synchronized Multimedia Integration Lan^age (SMIL) format, any 
JPEG format, any Graphics Interchange Format (GIF) , audio or 
digital audio formats^ Audio IFF, Computer Graphics Metafile, 
TIFF, BIPP, bmp. Clear, PITS, NPP, OPP, PCX, PMG, TGA, XBM, 

10 mod, any Moving Picture Bacperts Group (MPEG) format. Musical 
Instrument Digital Interface, PICT, PNG, Portable Dociament 
Format (PDF), Portable Network Graphics," Portable Pixmap, 
• progressive coding, Quicktime, RIFF, Self Extracting Archive, 
sequential coding; server-parsed HTML, sprite. Tagged Image 

15 Pile Format, targa, Targa Graphics Adaptor, thumbnail, wav, 

WebCQM, wireless bitmap, xpm or a different frame rate video. 

In an alternative embodiment of the present invention the media 
objects are created automatically or semi -automatically . 

20 

Defining the parameters 126 is achieved by the management unit. 
The parameters of the media objects may define media object 
type (audio, video or image) , technical format (as described 
above e.g. JPEG), alternative tasks (full view, close-up), 
25 terminal requirements (e.g. Nokia 6100 series) . In addition, 
textual matter or preview versions may be included in the 
media objects. 

When the management unit has received all necessary information 
30 regarding media objects associated with a specific media stream 
the management unit packages the media objects as a set of 
media objects during packaging 128. The packaging 128 comprises 
associating the media objects to a specific media stream so 
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that each media object are attached to the time line of the 
media stream by defining the availability of the media object 
in accordance with the time line of the media stream. This may 
be achieved by defining the key moment as elapsed time for the 
Btart of the media stream (e.g. media stream title, madia 
object identification tag, and media object title - second 
goal), defining the media object's availability prior, during 
and after broadcast of the media stream, as well as defining an 
additional time period during which the availability of a media 
object is announced but not available for transfer (e.g. 
advertised prior to broadcast) . 



When the packaging 128 is accomplished the management unit 
initiates a publishing 130 of the media object or the set of 
15 media objects so that the media object or set of media- objects 
are associated with the specific media stream. 

During the media stream broadcast the management unit controls 
the availability of the media object or the set of media 
objects by managing 132 availability in accordance with the 
pre-defined timing and the progression of the media stream 
broadcast, while taking in to account delays In the start of 
the broadcast and commercial breaks. 



in conjunction with controlling availability of the media 
object or set of media objects the management unit enables one 
or more terminals to access the available media objects by 
providing 134 the requesting one or more terminals to transfer 
any particular available media object. 

The providing step 134 further comprises presenting a user 
interface to the one or more terminals, which user interface 
lists the set of media objects, which may be generated by. the 
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managing device as a response to a request from the one or more 
terminals such as accomplished by a user of a terminal 
depressing a hotkey for capturing a media stream moment, as 
described with reference to figure 3. By depressing the hotkey 
5 the user navigates to a web page or starts a particular 

application. The user interface presents a specific list for a 
specific terminal, which specific list comprises media objects, 
which are readable by said specific terminal. Hence the user 
interface is dynamic in relation to the one or more terminals. 
10 For example, if the management unit knows that the specific 

terminal only supports PNG images the user interface does not 
present available GIF Images. 

Furthermore, the list may comprise presently unavailable media 
15 objects that will be available in the future and/ or previously 
available media objects, which presently are unavailable. 
Alternatively, if only one media object is available at any 
time the user interface may direct the user to directly 
transfer the media object omitting listing alternatives - 

20 

The user/users of the one or more terminals are subsequently 
asked whether a purchase of a media object is requested during 
a purchase? step 136, The media object may have a price, which 
subsequently to the user transferring the media object is 
25 charged to the user. If the user does not wish to purchase a 
media object, the method 120 is terminated in termination step 
138. 

On the other hand if the user wishes to purchase a media object 
30 of the list of media objects the method 120 moves to transfer 
the chosen media object during a transfer step 140. When the 
chosen media object is transferred to the user the transaction 
is recorded and processed by a separate transaction processing 
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device euch as the billing device as described with reference 
to figiire 3. The transaction may be accomplished in a wide 
variety of ways such as micro -payraent, charging against user 
account or operating billing. 

5 

When the media object is transferred to the user's terminal it 
may be identified by the terminal by its format or supporting 
application (e.g. through MIME type mapping) . This initial 
identification may further reveal which type of potential use 

10 is allowed. That is, additional rights and limitations may be 
attached to the media object (e.g. the user's ability to 
forward the media object to others may be limited) . Hence, when 
a specific user of a terminal purchases a certain media object^ 
which is transferred to the termiixal, the media object includes 

15 privileges describing rights and limitations in use or copying 
of the media object. 
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1, A system for delivering a media object to one or more 

5 terminals/ which media object ia associated with a media stream 
broadcasted to one or more media stream receivers/ said system 
comprising: 

(a) a broadcasting network for connecting to said one or 
more media stream receivers; 
10 (b) a first communications network for connecting to said 

one or more terminals; and 
(c) a communication device connecting to said broadcasting 
network and broadcasting said media stream to said one 
.or more media stream receivers, and connecting to said 
15 comimunications network and communicating said media 

object to said one or more terminals. 

2. A system according to claim 1, wherein said media stream 
coTiprises a television and/or radio transmitted shoW/ drama, 

20 movie/ sports game, news, or any combination thereof. 



3. A system according to claims 1 or 2, wherein said media 
object coTOprisee a text/ a picture, a series of pictures/ a 
video, a series of videos, an audio recording, a series of 

25 audio recordings, or any combination thereof. 

4, A system according to any of claims 1 to 3, wherein said 
terminal comprises a phone, a cellular or mobile phone/ a 
personal computer, television, a set top box/ a multimedia 

3 0 terminal/ a personal office assistant or any combination 
thereof . 
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5. A system according to any of claims 1 to 4, wherein said one 
or more media stream receivers comprise a set top box, 
multimedia terminal, television receiver, television, radio 
receiver or any combination thereof. 

6. A system according to any of claims 1 to 5, v;herein said 
communication device broadcasts to said one or more media 
stream receivers by a cable television network, a satellite 
television network^ a radio frequency television network, a 
telephone network, a powerline network, a radio network or any 
combination thereof. 

7. A system according to claim 6, wherein said comm\mication 
device is adapted to transmit digitally coded communication 

15 such as digital video baroadcasting and/or digital audio 
broadcasting . 

8. A system according to any of claims 1 to 7, wherein said 
first communications network comprises a telephone wire 

20 network, a cable television network, a powerline network, a 
computer network, a wireless telephone network, or any 
combination thereof. 

9. A system according to any of claims 1 to 8, wherein said 
25 communication device conprising a broadcasting unit for 

broadcasting said media stream to said one or more media stream 
receivers, a management unit for providing said media object to 
said one or more terminals, and a second communications network 
for interconnecting said broadcasting unit and said maiiagement 
30 unit. 

10. A system according to claim 9, wherein said broadcasting 
unit comprises a marker for generating a media stream 
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identification tag associated to said media stream, which media 
stream identification tag comprises information regarding 
duration of said media stream, lapsed time of said media 
stream, broadcasting schedule for said media stream, 
broadcasting channel for said media stream, or any combination 
thereof . 

11. A system according to claims 9 or 10, wherein said 
broadcasting unit is adapted to perform a continuous 
communication of data regarding said media stream information 
tag, an updating communication of revision of specific data 
regarding said media stream information tag, a communication 
based on schedule of said media stream, or any combination 
thereof . 

12. A system according to any of claims 9 to 11, wherein said 
management unit comprlseB an application program interface for 
receiving said media object, a database for storing said media 
object and said media stream identification tag, a real time 
publishing interface for enabling real time publishing of said 
media object, and a user interface for providing said one or 
more terminals access to select said media object stored in 
said database through said first communications network. 

13. A system according to claim 12, wherein said user interface 
xs adapted to respond to a request from said one or more 
terminals and to generate a media object list of media objects, 
which are accessible for said one or more terminals. 

14. A system according to any of claims 9 to 13, wherein said 
second communications network comprises local area network, 
metropolitan area network, wide area network, or inter-network 
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such as the Internet, a dedicated communication line, or any 
combination thereof. 

IS. A system according to any of claims a to 14 further 
5 comprises a billing unit for managing billing transactions for 
said one or more terminals' requests for said media object and 
for generating invoices to said one or more terminals in 
accordance with said billing transactions. 

10 16- A system according to claim 17 further comprises a third 
communications network for interconnecting said billing lanit 
and said management unit and a fourth communications network 
for interconnecting said billing unit and said one or more 
tesrrainals. 

15 

17. A system according to claim 16, wherein said third and 
fouit:h communications network comprises local area network, 
metropolitan area network, wide area network, or inter-network 
such as the Internet, a dedicated communication line, a 

20 telephone wire network, a cable television network, a powerline 
network, a conqputer network, a wireless telephone network, or 
any combination thereof. 

18. A system according to any of claims 1 to 17, wherein said 
25 communication device broadcasting said media stream and said 

media object through said broadcasting network and wherein said 
one or more media stream receivers connecting to said first 
commvinications network and communicating said media object to 
said one or more terminals. 

30 

19. A method for delivering a media object to one or more 
terminals, which media object is associated with a media stream 
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broadcasted to on^ or more media stream receivers, and said 
method comprising: 

(a) associating said media object with said media stream by 
means of a communication device; 
5 (b) broadcasting said media stream to said one or more 

media stream receivers through a broadcasting network 
by means of said communication device; and 
(c) communicating said media object to a requesting 
tearminal of said one or more terminals through a 
10 comimunications network by means of said communication 

device • 

20. A method according to claim 19 further comprises defining a 
parameter for said media object by means of said communication 
15 device and said parameter defining a media object format such 
as audio, video, image, or any combination thereof, a technical 
format, an alternative task such as full view or close-up, a 
terminal requirement, or any combination thereof - 

20 21. A method according to claims 19 or 20 further comprises 
packaging a set of media objects associated with said media 
stream and publishing said set of media objects to said one or 
more terminals by means of said comrminication device. 

25 22. A method according to claim 21, wherein said packaging 
comprises linking said media object to said media stream so 
that said media object is attached to a broadcasting time line 
of said media stream and defining the availability of said 
.media object in accordance with said broadcasting time line of 

30 said media stream. 

23, A method according to claims 21 or 22, wherein said 
packaging fxirther con^rises defining a media object based on a 
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key moment of said media stream as an elapsed time from the 
start of the media stream, defining the media object's 
availability prior, during and after broadcast of said media 
stream, defining an additional time period during which said 
media object's availability is announced but not available for 
transfer, or any conOaination thereof. 

24. A method according to any of claims 19 to 23 further 
comprises managing said set of media objects by means of said 
communication device, and said managing comprises controlling 
availability of each media object of said set of media objects 
in accordance with said broadcast time line for said media 
stream. 

15 25. A method according to claim 24 further conpriees providing 
said one or more terminals access to said available media 
objects and enabling a requesting terminal of said one or more 
terminals to transfer any specific available media object. 



10 



20 



25 



26. A method according to claim 25, wherein said providing 
comprises presenting an interface to said one or more 
terminals, which interface lists said set of media objects. 

27. A method according to any of claims 19 to 26 further 
con5>ris6s generating a media object by means of said 
communication device in response to a request from said one or 
more terminals. 

28. A method according to claim 27, wherein said request is 
30 accomplished by a user of a terminal depressing a hotkey for 

capturing a key moment of said media stream. 
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29. A method according to any of claims 26 to 28, wherein aaid 
interface presents a specific list for a specific terminal/ 
which specific list comprises a media object, which is readable 
by said specific terminal. 

5 

30. A method according to any of claims 19 to 29 further 
comprises purchasing said media object from said communication 
device by means of said one or more terminals, by purchasing 
said media object said media object is transferred to said one 

10 or more tertainals. 

31. A method according to claim 30 further comprises recording 
and processing of said transfer of said media object to said 
one or more terminals by means of a transaction processing 

15 device • 

32. A method according to claims 30 or 31 further comprises 
identifying said media object format by means of said one or 
more terminals, said identifying revealing information such as 

20 supporting application needed, additional rights pertaining to 
said media object, forwarding limitations associated with said 
media object, or any combination thereof. 

33. A method according to any of claims X9 to 32, further 
25 . comprising providing privileges associated with said media 

object and with said one or more terminals, which privileges 
enable said one or more terminals to copy or forward said media 
object in accordance with each of said one or more terminals' 
number of purchases of said media object and/or which 
3 0 privileges disable said one or more terminals to copy or 

forward said media object and/or which privileges disable said 
one or more terminals to copy or forward more than said one or 
more terminals' number of purchases » 
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34. A method according to any of claims 19 to 33, wherein said 
method further incorporates any features of the system 
according to ^y claims 1 to 18. 
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Abstract 



This Invention relates to a system and method for delivering a 
media object associated with a' media stream broadcasted from a 
conmtunication device to a broadcast receiving unit such as a 
personal computer, a multimedia terminal, a television 
receiver, a television, or any type of radio receiver, to a 
terminal such as a phone, a cellular or mobile phone, a 
personalcomputer, a television, or a personal office assistant 
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